Image Generation System

ABSTRACT

An image generation system for generating an image on a display screen. The image generation system includes an eye-tracking system capable of determining a user&#39;s eye orientation and outputting a signal indicative of same. The image generation system also includes a bio-feedback sensor capable of detecting activity of one or more physiological functions of the user and outputting a signal indicative of the level of activity. A processor is included and is adapted to receive and process the output signals from the eye-tracking system and bio-feedback sensor. The processor determines an image to be generated on the display screen indicative of the signals from the eye-tracking system and bio-feedback sensor.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of PCT application numberPCT/NZ2008/000212, filed Aug. 15, 2008, the entire contents of which areincorporated herein by reference, which claims the benefit of NewZealand patent application serial number 560457 filed Aug. 15, 2007 andNew Zealand patent application serial number 560457 filed Jul. 14, 2008.

TECHNICAL FIELD

The present invention relates to an image generation system.

In particular, the present invention relates to an image generationsystem and method using bio-feedback and eye-tracking systems.

BACKGROUND ART

The generation of pictures, text and other images is an importantcomponent in human communication. Images provide a permanent orsemi-permanent mode of communication capable of relaying a great amountof information in a small timeframe.

Images are also one of the fundamental expressions of human creativityand may be used to convey emotions, thoughts, information and new formsof understanding.

Artists are continually adapting tools and techniques to create newtypes of images and modes of expression. For example, developments incomputer animation technology have transformed film-making and graphicdesign.

Images may be generated on a piece of physical material (e.g. paper,canvas etc.) or via electronic displays. To generate images a personmust manually draw or paint the image using a pencil, pen, brush orother drawing tool, or on an electronic display via a suitablyprogrammed computer, a computer mouse and/or other input device. Otherdisplays may use images captured via a camera or an electromagneticimage capture device or generated by an algorithm specifically designedto create images.

The control of computers via electronic display screens has rapidlydeveloped from keyboards to include mice, pens, touch-screens and otherinput devices.

During interaction with a graphical user interface (GUI) such devicescan be used to control the computer functions in addition to generatingimages if required.

An intrinsic requirement of such image generation systems is the needfor manual input/manipulation by the user to draw an image.

The ‘quality’ of such an image is thus directly influenced by both theartist's talent and technical ability in manipulating the image.

Users whose hands (or other limbs) are occupied, constrained,restrained, paralyzed or disabled are clearly impeded from maneuvering apen or interface device without assistance and are thus hampered fromcontrolling a computer, or drawing an image.

To obviate the need for manual interaction with a movable control,various systems have been developed and generally fall into twocategories, namely eye-tracking and bio-electrical sensor based systems.

Many known eye-tracking systems are capable of tracking the movement ororientation of a person's eyes to determine the direction of the user'sgaze and to control a device accordingly, e.g. known weapon aimingcontrol systems on aircraft may use eye-tracking systems to determinewhere a pilot is looking and, accordingly aim a slaved weapon.

Further prior art developments on basic eye-tracking systems have usedeye-blinks, saccade and other movement to control functions of acomputer to provide a control interface for paralyzed patients and thelike.

Known bio-electrical control systems include Brain-Computer Interfaces(BCIs) that include electrodes connected to portions of the brain tocontrol devices such as cameras, artificial limbs, control systems orthe like. Some BCIs may also receive external signals and convert toelectrical impulses passed to the brain to simulate normal sensorysystems.

For example, an artificial ear may include a microphone coupled to aprocessor linked to electrodes in the auditory parts of the brain of auser. The processor is capable of converting the microphone input toappropriate electrical signals to pass to the brain thereby providingthe user with hearing ability.

Another form of bio-electrical control system is that used in thecomputer-based meditation system The Journey to Wild Divine by Smith.

The system devised by Smith uses bio-feedback (i.e. heart-rate and skinconductivity) from sensors placed on a user's fingers to solve problemsand complete tasks set by a computer program and displayed on a screen.For example, one such task sets a heart-rate level, below which the usermust lower their heart-rate to move onto the next task.

Another biofeedback system is described in U.S. patent application Ser.No. 10/028,902 (published as US 2002/0077534) by DuRousseau. TheDuRousseau system uses biofeedback from multiple physiological sourcesto effect a control interface with a computer. The DuRousseau systemhowever does not track where the user is looking.

An example of a system that combines the eye-tracking functionality andbio-electrical feedback is disclosed in U.S. Pat. No. 5,649,061 bySmyth, the entire contents of which is herein incorporated by reference.

Smyth describes the use of an eye-tracking system combined with anelectronic bio-electric signal processor and digital computer todetermine the viewer's eye-fixation and determine a mental decision fromcorresponding event-evoked cerebral electric potential. Thus, a user cancontrol a device by using eye-tracking to set a point of interest orfunction and a threshold cerebral electric potential to act as a switchor control.

While the Smyth system provides an effective system for controllingmachines or the like, Smyth does not describe any way in which thesystem could be used for generation or control of images.

It would thus be advantageous to provide an image generation systemcapable of generating or controlling an image by using an interface thatdoes not require physical manual manipulation of a control device.

It is an object of the present invention to address the foregoingproblems or at least to provide the public with a useful choice.

All references, including any patents or patent applications cited inthis specification are hereby incorporated by reference. No admission ismade that any reference constitutes prior art. The discussion of thereferences states what their authors assert, and the applicants reservethe right to challenge the accuracy and pertinency of the citeddocuments. It will be clearly understood that, although a number ofprior art publications are referred to herein; this reference does notconstitute an admission that any of these documents form part of thecommon general knowledge in the art, in New Zealand or in any othercountry.

It is acknowledged that the term ‘comprise’ may, under varyingjurisdictions, be attributed with either an exclusive or an inclusivemeaning. For the purpose of this specification, and unless otherwisenoted, the term ‘comprise’ shall have an inclusive meaning—i.e. that itwill be taken to mean an inclusion of not only the listed components itdirectly references, but also other non-specified components orelements. This rationale will also be used when the term ‘comprised’ or‘comprising’ is used in relation to one or more steps in a method orprocess.

Further aspects and advantages of the present invention will becomeapparent from the ensuing description which is given by way of exampleonly.

DISCLOSURE OF INVENTION

According to a first aspect of the present invention there is providedan image generation system for generating an image on a display screenor the like, the image generation system including:

-   -   an eye-tracking system capable of determining a user's eye        orientation and outputting a signal indicative of same,    -   a bio-feedback sensor capable of detecting activity of one or        more physiological functions of the user and outputting a signal        indicative of the level of activity, and    -   a processor adapted to receive and process the output signals        from the eye-tracking system and bio-feedback sensor,

wherein the processor determines an image to be generated on the displayscreen indicative of the signals from the eye-tracking system andbio-feedback sensor.

Preferably the eye-tracking system is also capable of detectingeye-movement of a user.

Preferably the processor is capable of processing the signals from theeye-tracking system to determine the time-duration in which a user'seyes are directed in one or more viewing directions from the detectedeye-orientation, hereinafter termed an “eye-fixation”. It should beappreciated that the time-duration may be measured as:

-   -   an average time-duration of eye-fixation in a particular viewing        direction during a predetermined time-period;    -   the time-duration of eye-fixation in a particular viewing        direction as a proportion of a predetermined time-period; and/or    -   the time-duration of eye-fixation in a particular viewing        direction during saccadic eye-movement.

Preferably the processor determines a location to display a part of thegenerated image from the output signals from the eye-tracking system.For example, in one embodiment, the position of the image (or partthereof) generated on the display screen may correspond to the user'seye-fixation in a particular direction.

Preferably the processor varies at least one parameter of a part of theimage to be displayed on the display screen from the output signals fromthe bio-feedback sensor. For example, in one embodiment, increases in auser's heart-rate may invoke a commensurate increase in a parameter,e.g. brightness, of the image generated on the display screen.

In one embodiment the processor may determine a sound to be generatedfrom a speaker indicative of the signals from the eye-tracking systemand/or bio-feedback sensor. Thus, not only is an image generated, butalso a sound.

In one embodiment the eye-tracking system and bio-feedback sensor may beadapted to be substantially portable and capable of remotelytransmitting a signal to the processor.

According to another aspect of the present invention there is provided amethod of image generation using an image generation systemsubstantially as hereinbefore described, said method including;

-   -   presenting a first image to a user,    -   determining one or more points of eye-fixation of a user on the        first image with said eye-tracking system,    -   said processor determining an image to display on the display        screen indicative of the signals from the eye-tracking system        and bio-feedback sensor.

Preferably the method substantially as hereinbefore described isimplemented by one or more computer systems programmed with computerexecutable instructions.

Reference herein to a ‘parameter’ of an image or part thereof includes,but is not limited to, one or more of the: size, color, contrast,brightness, saturation, edge-contrast, hue, pitch, resolution,time-duration of display.

The ‘image’ displayed on the display screen is preferably a visualrepresentation. It will be appreciated that the image may include anyvisual representation of abstract or real objects, ideas, features orinformation and may include text, symbols and the like.

In some embodiments the ‘image’ may take the form of a ‘scientificvisualization’, i.e. the representation of data as an image.

The ‘display screen’ as referred to herein includes any form of displaycapable of producing an image from signals sent by the processor, and byway of example, may include one or more Cathode Ray Tube (CRT), LiquidCrystal Displays (LCD), plasma displays, projectors, virtual retinaldisplays (VRD) projection displays, Light Emitting Diode (LED) displaysor the like.

As used herein, the term ‘bio-feedback’ refers to a measurablebiological metric indicative of the activity of a physiological functionof a user.

A physiological function preferably includes the Autonomic NervousSystem (ANS).

As used herein, the term ‘signals’ refers to any form of signal andincludes, but is not limited to, one or more electrical, magnetic, orelectromagnetic signals.

Many eye-tracking systems are known and use various methods fordetermining eye-movement and viewing direction and are not explainedfurther herein. Examples of known eye-tracking systems may be found inU.S. Pat. Nos. 5,638,176, 5,331,149, 5,430,505, 4,720,189, though thislist is by no means comprehensive and many eye-tracking systems andsensors are known and capable of being used with the present invention.For example, an exemplary eye-tracking system that may be used with thepresent invention may be the Tobii™ eye-tracking system produced byTobii TechnologyAB.

For the purposes of the present invention, the eye-tracking system ispreferably capable of detecting the orientation of the eye and thus theviewing direction or ‘line-of-sight’ of the user. Preferably theeye-tracking system is also capable of detecting eye-movement of a user.

Preferably the eye-tracking system includes an oculometer capable ofdetecting the orientation of an eye.

Preferably the oculometer is movable on a mounting such that it may bealigned with a user's eyes. For example, the oculometer may be mountedto be rotatable and/or merely moveable about or along vertical andhorizontal axes.

In a further embodiment the oculometer may be automatically movable tomaintain alignment with a user's eyes.

In yet another possible embodiment, the oculometer may be coupled to auser-location system, the user-location system capable of identifyingthe location and/or orientation of the user and determining the positionof the user's eyes by extrapolating from the user location. Such a‘user-location’ system may include any form of object locationincluding: digital cameras and image-recognition software, laser scannerinterfaces, optical motion capture systems or the like.

The bio-feedback system preferably includes, but is not limited to, oneor more sensors capable of measuring galvanic skin response, heart-rateand blood pressure, cerebral electric potential, skin temperature,muscle tension, heart-rate variability, blood-pressure variability orskin moisture of a user.

It will be appreciated that the ‘processor’ referred to herein mayinclude multiple individual processors for processing the output signalsfrom each of the eye-tracking system and bio-feedback sensor and fordetermining an image to display on the display screen.

In one preferred embodiment, a processor is provided for each of theeye-tracking system and the bio-feedback sensor and are programmed toprocess the output signals from the eye-tracking system and bio-feedbacksensor for transfer to a digital computer, the digital computerincluding a processor capable of determining an image to be generated onthe display screen indicative of the signals from the eye-trackingsystem and bio-feedback sensor.

According to another aspect of the present invention there is providedan image generation system for generating an image on a display screenor the like, the image generation system including:

-   -   a first display having the display screen,    -   an eye-tracking system capable of determining a user's eye        orientation and outputting a signal indicative of same,    -   a processor adapted to receive and process the output signals        from the eye-tracking system, and    -   a second display for presenting a first image to a user,

wherein the eye-tracking system is capable of determining one or morepoints of eye-fixation of a user on the first image, the processordetermining a second image to display on the first display indicative ofthe signals from the eye-tracking system.

In one embodiment the second display may include physical objects orimages, such as people, animals, plants, inanimate objects, paintings,drawings, or other artworks, the first image being formed therefrom.

It will be appreciated that where the second display includes suchphysical objects or images, an image capture device e.g. a video orstill camera, may be required to convert the physical first image intosignals indicative of same, such that those signals may be processed.

Preferably the processor determines a position of a part of the secondimage to be displayed on the first display from the output signals fromthe eye-tracking system. For example, in one embodiment, the position ofthe parts of the second image generated on the first display maycorrespond to the user's eye-fixation in a particular direction on thefirst image.

The part of the first image fixated on may be defined as anarea-of-interest e.g. an area of the first image that the user fixateson for a relatively greater time than other areas, or an area whichevokes a change in bio-feedback response, indicating a higher level of‘interest’ in that area.

In a further embodiment, the processor also determines a parameter of apart of the second image to be displayed on the first display from theoutput signals from the eye-tracking system. For example, in oneembodiment, the scale of the second image generated on the first displaymay be proportional to the time-duration of a user's eye-fixation on aparticular point of the first image.

In another embodiment the processor may determine a parameter of a partof the second image to be displayed on the first display from the outputsignals from the eye-tracking system indicative of a level of saccade.For example, in one embodiment, the brightness of the second imagegenerated on the first display may increase if the eye-tracking systemdetects a predetermined level of rapid saccadic eye-movements.

Preferably the image determined to be displayed on the first display isan at least partial copy or representation of the first image.

In a further embodiment, the image determined to be displayed on thefirst display is an at least partial copy or representation of a part ofthe first image corresponding to the user's eye-fixation on the firstimage.

In one embodiment the size of the part of the first image copied anddisplayed on the first display is dependant on the output signals fromthe eye-tracking system.

For example, in one embodiment, the extent of saccadic eye-movement of auser may define a particular area-of-interest of the first image whichis then copied to the first display. Alternatively, the size of the partof the first image ‘copied’ may be proportional to the time-duration ofeye-fixation on a particular point of the first image.

In one preferred embodiment the second display may include a seconddisplay screen, the first image being displayed thereon.

In an alternative embodiment the first and second displays may be partsof a single display screen, e.g. a split-screen display.

In a further embodiment the first image may be a three-dimensional imageof a virtual-reality environment, wherein a control system is providedto navigate through the virtual-reality environment and thus change thefirst image.

In a yet further embodiment, the control system may include theeye-tracking system and bio-feedback sensor. Thus the image generationsystem may also function as a control system for manipulating a virtualenvironment.

Preferably the processor is connected to the second display andprogrammed to present the first image as filtered, processed, distortedand/or rendered at least partially unclear. For example, the first imagemay be faded, diffuse, dimmed, or otherwise unclear.

In a further embodiment the processor is programmed to vary at least oneimage parameter the first image to improve image clarity (e.g. byincreasing brightness) when the eye-tracking system detects the userlooking at a said part of the first image. Thus, by looking at variousparts of an ‘unclear’ first image the user may make clear those parts.

According to yet another aspect of the present invention there isprovided an image generation system for generating an image on a displayscreen or the like, the image generation system including:

-   -   an eye-tracking system capable of determining a user's eye        orientation and outputting a signal indicative of same,    -   at least one bio-feedback sensor capable of detecting activity        of one or more physiological functions of the user and        outputting a signal indicative of the level of activity,    -   a processor adapted to receive and process the output signals        from the eye-tracking system and bio-feedback sensor, and    -   a second display for presenting a first image to a user,

wherein the eye-tracking system is capable of determining one or morepoints of eye-fixation of a user on the first image, the processordetermining a second image to display on the first display indicative ofthe signals from the eye-tracking system and bio-feedback sensor.

Preferably the processor determines a position of a part of the secondimage to be displayed on the first display from the output signals fromthe eye-tracking system. For example, in one embodiment, the position ofthe second image generated on the first display may correspond to theuser's eye-fixation in a particular direction on the first image.

Preferably the processor determines a parameter of a part of the secondimage to be displayed on the first display from the output signals fromthe bio-feedback sensors. For example, in one embodiment, the scale ofthe image generated on the first display may be proportional to theuser's blood-pressure.

In another embodiment the processor may determine a parameter of a partof the image to be displayed on the first display from the outputsignals from the eye-tracking system indicative of a level of saccade.For example, in one embodiment, the brightness of the image generated onthe first display may increase if the eye-tracking system detects apredetermined level of rapid saccadic eye-movements.

Preferably the image determined to be displayed on the first display isan at least partial copy or representation of the first image.

In a further embodiment, the image determined to be displayed on thefirst display is an at least partial copy or representation of a part ofthe first image corresponding to the user's eye-fixation on the firstimage.

It will be appreciated that the processor may be adjusted to link anyparticular parameter of the second image displayed with any particularbio-feedback parameter. For example, in one embodiment, where a part ofthe first image invokes strong ‘interest’ (i.e. an area-of-interest)from the user (as detected by the bio-feedback sensors), the copy of thefirst image part may be displayed slightly smaller and with full color.In another embodiment, where the user merely glances at a first imagepart, the copy’ may be slightly larger, more diffuse, and, lacking incolor i.e. gray scale.

It will be appreciated that numerous copying configurations are possibleand the image parts displayed on the first display may be any shape andsize and need not correspond to the first image part copied.

Preferably the size of the part of the first image ‘copied’ and thendisplayed on the first display is dependant on the output signals fromthe bio-feedback sensor. For example, in one embodiment a user may viewa first image and parts of that first image are ‘copied’ to the firstdisplay, the size of the part copied proportional to the galvanic skinresponse of a user.

In an alternative embodiment, the size of the part of the first imagecopied and displayed on the first display is dependant on the outputsignals from the eye-tracking system. For example, in one embodiment theextent of saccadic eye-movement of a user may define a particular‘area-of-interest of the first image which is then copied to the firstdisplay to form the second image.

In preferred embodiments the second display may be a second displayscreen, the first image being displayed thereon.

In a further embodiment the first image may be a three-dimensional imageof a virtual-reality environment, a control system provided to navigatethrough the virtual-reality environment and thus change the first image.

In another embodiment, the second display may include physical objectsor images, such as people, animals, plants, inanimate objects,paintings, drawings, or other artworks, the first image being formedtherefrom.

It will be appreciated that where the second display includes suchphysical objects or images, an image capture device e.g. a video orstill camera, may be required to convert the physical first image intosignals indicative of same, such “that those signals may be processed.

The present invention may thus provide an improved image generationsystem, capable of generating an image on a screen based on eye-movementand bio-feedback of a user.

According to yet another aspect of the present invention there isprovided a communication system for allowing communication between usersat two separate user locations, the communication system including twoimage generation systems, each said image generation system operable togenerate an image on a viewer display screen and each image generationsystem including:

-   -   an eye-tracking system capable of determining a user's eye        orientation and outputting a signal indicative of same,    -   bio-feedback sensor capable of detecting activity of one or more        physiological functions of the user and outputting a signal        indicative of the level of said activity,    -   a processor adapted to receive and process the output signals        from the eye-tracking system and bio-feedback sensor, and    -   a second display for presenting a first image to the user, the        second display connected to an image capture device to capture        said first image,

wherever the eye-tracking system is capable of determining one or morepoints of eye-fixation of a user on the first image, the processordetermining an image to display on the display screen indicative of thesignals from the eye-tracking system and bio-feedback sensor, and

at each of said user locations, the bio-feedback sensor, eye-trackingsystem, processor and second display screen of one image generationsystem, and the viewer display screen and image capture device of theother said image generation system are located, and

-   -   each image capture device is configured to capture an image and        display said captured image on the second display at the other        location,    -   each processor is configured to determine an image to display on        the viewer display screen at the other location indicative of        the signals from the eye-tracking system and bio-feedback        sensor.

The image capture device is preferably a video camera adapted to capturean image of one of the users at the respective location. It will beappreciated that an audio capture device may also be used to transferaudio between the users at the separate locations.

Reference herein to a “video camera” should be understood to include alldevices capable of capturing a still or video image and includes digitaland analog devices.

Thus, each user may communicate by viewing a video-feed of the otheruser while also simultaneously viewing an image generated on anotherscreen of an image generated by the signals from the eye-tracking systemand bio-feedback sensor of the other user.

According to another aspect of the present invention there is provided asound generation system for generating a sound from a speaker or thelike, the sound generation system including:

-   -   an eye-tracking system capable of determining a user's eye        orientation and outputting a signal indicative of same,    -   at least one bio-feedback sensor capable of detecting activity        of one or more physiological functions of the user and        outputting a signal indicative of the level of activity, and    -   a processor adapted to receive and process the output signals        from the eye-tracking system and bio-feedback sensor,

wherein the processor determines a sound to be generated from thespeaker indicative of the signals from the eye-tracking system andbio-feedback sensor.

In a further embodiment the processor also determines an image to begenerated on a display screen indicative of the signals from theeye-tracking system and bio-feedback sensor.

The present invention may thus provide an improved sound generationsystem, capable of generating a sound from a speaker based oneye-movement and bio-feedback of a user.

According to yet another aspect of the present invention there isprovided a method of image generation including:

-   -   presenting a first image to a user;    -   determining the user's eye-orientation using an eye-tracking        system;    -   measuring the activity of one or more physiological functions of        the user using a bio-feedback system;    -   generating a second image indicative of the user's eye        orientation and physiological function(s).

It should be appreciated that the first image may be actively presentedto the user e.g. via a display screen, or alternatively may form part ofany physical or virtual environment capable of being viewed by a user.

According to a further aspect of the present invention, there isprovided a method of assessing a user's response to a presented image,the method including:

-   -   presenting a first image to a user;    -   determining the user's eye-orientation using an eye-tracking        system;    -   measuring the activity of one or more physiological functions of        the user using a bio-feedback sensor;    -   generating a second image indicative of the user's eye        orientation and physiological function(s);    -   correlating parts of the image identified as areas-of-interest        with the bio-feedback measured when the user fixates on the        areas-of-interest.

Thus, this method may be used to measure a physiological manifestationof a user's emotional response to an image presented to the user.

According to a further aspect of the present invention, there isprovided a method of image generation including:

-   -   a) presenting a first image of a virtual environment to a user;    -   b) determining the user's eye-orientation using an eye-tracking        system;    -   c) measuring the activity of one or more physiological functions        of the user using a bio-feedback sensor;    -   d) changing said first image presented to the user to present a        new ‘second’ image, the second image being indicative of the        user's eye orientation and physiological function(s) when        viewing the first image.

Preferably, steps a)-d) of the aforementioned method are performediteratively. Thus, the user may be provided with the ability to interactwith the virtual environment, e.g. as in a computer game or educationalsoftware. The image presented to the user may thereby be based on wherethey look in the virtual environment and their physiological response towhere they look.

Preferably, the image presented to the user is indicative of the virtualenvironment surrounding and/or proximal a virtual entity, e.g. anavatar's view.

As used herein the term ‘avatar’ refers to a virtual representation of aperson, or a game character or other virtual entity capable of beingcontrolled by the user in a virtual environment, e.g. a computer gameenvironment.

The sources for images generated and processed by the image generationsystem may range from static images such as computer interfaces,photographs, graphic designs, advertisements, physical environments orimages of dashboards, to real-time viewing of video inputs from anysource whether computer generated or video captured.

According to another aspect of the present invention, a said display maybe a three-dimensional display device such as a ‘3D’ printer, robot,digital fabricator or the like. Thus, the ‘image’ generated may be aphysical three-dimensional object indicative of the signals receivedfrom the bio-feedback sensor and eye-tracking system.

According to yet another aspect of the present invention, the seconddisplay may be a virtual three-dimensional element, the processorconfigured to alter the three-dimensional characteristics of saidelement to a form indicative of the signals received from thebio-feedback sensor and eye-tracking system.

In one embodiment, the three-dimensional element is an ‘avatar’ or thelike in a virtual environment.

Thus, the image generated may not necessarily be immediately visible asthe image generated is an alteration of a 3D element and may only bevisible when viewing the element at a particular virtual ‘angle’,‘perspective’ ‘lighting’ or the like.

In another embodiment, the three-dimensional element is an aspect of‘lighting’ within the virtual environment.

According to yet another aspect of the present invention the imagegeneration system further includes at least one motion sensor capable ofdetermining movement of the user and outputting a signal indicative ofsame, processor capable of determining an image to be generated on asaid display indicative of the signals from the eye-tracking system,bio-feedback sensor and motion sensor.

It should be appreciated that in another aspect, any one of the methodsas aforementioned may be implemented by computer program instructionsstored on a computer-readable medium, e.g. computer memory, disc, RAM orROM.

The present invention may thus be used as a control in a virtual realityenvironment for manipulating virtual objects and/or entities within thevirtual environment through use of a combination of eye-tracking,bio-feedback and motion sensing.

The present invention may thus provide an image generation systemcapable of generating an image indicative of a user's eye-fixation andphysiological response when viewing an image.

BRIEF DESCRIPTION OF DRAWINGS

Further aspects and advantages of the present invention will becomeapparent from the following description which is given by way of exampleonly and with reference to the accompanying drawings in which:

FIG. 1 shows a schematic diagram of an image generation system accordingto a first preferred embodiment of. the present invention;

FIG. 2 shows a schematic diagram of an’ image generation systemaccording to a second preferred embodiment of the present invention;

FIG. 3 shows a process diagram of a method of operating the imagegeneration system shown in FIG. 1 or 2;

FIG. 4 shows a system diagram of the image generation system shown inFIGS. 1 and 2;

FIG. 5 shows a system diagram of an image generation system according toa second preferred embodiment of the present invention, the imagegeneration system having a single display;

FIG. 6 shows a process diagram of a computer game for use with the imagegeneration system of FIG. 5;

FIG. 7 shows two screenshots of another computer game for use with theimage generation system of FIG. 5;

FIG. 8 shows the first and second display screens of the imagegeneration system of FIGS. 1-4;

FIG. 9 shows another embodiment of the present invention, implemented asa computer game.

BEST MODES FOR CARRYING OUT THE INVENTION

FIG. 1 shows an image generation system (1) according to one preferredembodiment of the present invention. The image generation system (1) hasa ‘first’ display screen (2) for displaying an image (12) indicative ofthe signals from an eye-tracking system (3) and bio-feedback sensors(4).

The eye-tracking system (3) has an oculometer (5) capable of determiningthe eye-orientation of a user (6) and outputting a signal indicative ofsame. The oculometer (5) thus allows the viewing direction or‘eye-fixation’ (14) of the user (6) to be determined.

The bio-feedback sensors (4) are provided as sensory pads (7) attachedto the fingers (8) of the user (6) and form part of a bio-feedbacksystem (16). The sensory pads (7) are capable of outputting a signaldependant on the heart-rate, galvanic skin response or otherphysiological function of the user (6).

The image generation system (1) has a digital signal processor (9)programmed to receive and process the output signals from theeye-tracking system (3) and bio-feedback sensors (4). The processor (9)forms part of a computer system such as a personal computer (PC) (notshown). The image generation system (1) is integrated with the computersystem and provides an interface for the user (6). The computer programfor controlling the processor (9) is stored on a computer-readablemedium such as a computer Hard Disk Drive, CD/DVD-ROM, Solid-statememory device or similar.

A second display is provided in the form of second display screen (10)for presenting a first image (11) to the user (6).

The image generation system (1) also incorporates a speaker (17) that isconfigured to generate a sound, thus adding an additional stimulus forthe user (6). The processor (6) can be configured to monitor the soundfrom the speaker (17) for correlation with the bio-feedback (108) andeye-tracking (104) responses. Monitoring the user's responses to bothsound and images can provide additional information about the user (6)and the correlation between different sounds and images. Alternatively,the processor (9) can be configured to output a sound indicative of thesignals from the eye-tracking (3) and/or bio-feedback (16) systems.

The eye-tracking system (3) is capable of determining one or more pointsof eye-fixation of the user (6) on the first image (11), the processor(9) then determining the second image (12) to display on the displayscreen (2) indicative of the signals from the eye-tracking system (3)and bio-feedback sensors (4).

The image (12) displayed on the display screen (2) is a composite imageformed by ‘copying’ parts (13) of the first image (11) corresponding tothe user's eye-fixation i.e. the parts (13) of the first image (11)which the user (6) looks at during saccadic eye-movement, or stares atfor a predetermined period of time. These parts can be considered‘areas-of-interest’ i.e. areas in which the viewer is most interested.

The parameters, e.g. size, intensity, contrast, clarity, coherence andcolor of the part (13), of the first image (11) copied and displayed onthe first display screen (2) as second image (12) are indicative of theoutput signals from the bio-feedback sensors (4), e.g. the size of thepart (13) may be proportional to the heart-rate of the user (6), whilethe intensity and color is dependant on the galvanic skin response.

The processor (9) determines a coordinate location of a part (13) of thefirst image (11) to be copied and displayed on the first display screen(2) from the output signals from the eye-tracking system (3), i.e. thelocation of the second image (12) generated on the first display screen(2) corresponds to the user's eye-fixation in a particular direction onthe first image (11).

Each user has a different relative eye-position and size and thus thesystem (1) will require calibration for each new user.

The calibration procedure, according to one preferred embodiment, is asfollows:

-   -   a) the user (6) wears a headpiece (not shown) having reference        transmitters (not shown) attached to either side of their head        which communicate with receivers (not shown) to locate the head        position of the user (6);    -   b) the oculometer (5) is aligned with the user's eyes to detect        the pupils;    -   c) the relative spatial separation of the user's eyes and        reference transmitters are then stored in a memory store (not        shown);    -   d) the processor (9) is programmed to move the oculometer to        maintain alignment with the eyes.

The system (1) is thus calibrated such that the receivers will indicatemovement of the user's head and the oculometer (5) will realignaccordingly based on the stored value of the spatial separation.

The viewing direction or eye-fixation (14) can then be calculated usingsuitable algorithms from the position of the user's head and orientationof the eyes.

It will be appreciated by one skilled in the art that the computerprogram of the present invention may be constructed to suit theapplication. However, in general, the present invention will use acomputer program that incorporates bio-feedback and eye-tracking signalprocessing algorithms and one or more image generation algorithms.

The bio-feedback algorithm compares the digital signals from thebio-feedback (4) sensors with an index of physiological indicator valuesand then calculates temporal bio-feedback data indicative of the user'sphysiological state.

Similarly, the eye-tracking algorithm processes the signals receivedfrom the oculometer. The eye-tracking algorithm compares the digitalsignals with a spatial index representing location references of pixelsdisplayed on the first (11) and/or second (12) images. The eye-trackingalgorithm then produces temporal eye-tracking data matching the locationof the pixels or general area of the image (11 or 12) that is beingviewed by the user (6).

The processor (9) processes the bio-feedback and eye-tracking data withan image generation algorithm that uses pre-defined rules (or anadaptive rule system) to output image data that is used by a GraphicsProcessing Unit (GPU) to alter the original image (11) or generateanother image (12). The image generated (11 or 12) is thus a temporalrepresentative of the direction the user (6) is looking and thebio-feedback response of the user at that time.

The computer program is constructed as a module of a Source DevelopmentKit (SDK) that provides software developers with an interface with thebio-feedback and eye-tracking systems and on which the developer canbuild a program utilizing the bio-feedback and eye-tracking data output.

It should be appreciated that a communications system is also capable ofbeing formed by communicatively coupling two said image generationsystem (1) together, each system (1) including first (2) and second (10)display screens with the first display screen (2) of each system (1)acting as the second display screen (10) for the other system. Videocameras capture an image of each user for display to the other user onadditional screens, or a split screen display. The image (11) presentedto each user is thus formed by what the other user sees and theirphysiological response to what they see.

With reference to FIG. 2, a further preferred method of operating theimage generation system (1) is now described.

After a user (6) enters a darkened room environment containing the imagegeneration system (1), having a second display screen (10), the system(1) is calibrated using the above calibration procedure.

After the system (1) is calibrated, first image (11) is displayed on thesecond display screen (10). Using control buttons (not shown), the usercan control the images (11) displayed and cycle through a sequence ofimages (11). The images (11) may be a static set of pre-selected imagesor a video feed. The image generation system (1) can also be used toselect pre-processed images where the range of images is restricted toimages from an ‘array’ or a ‘stack’. In this case, “portions” of imagesmight be referenced in real-time using eye-tracking (104) whilebio-feedback (108) is used to determine which image from the restrictedarray or stack the “portion” is drawn from.

The images (11) are initially displayed as dimmed or otherwise difficultto see, but as the user (6) tries to look at the images (11), theynotice ‘areas-of-interest’, i.e. the areas of the image (11) where theeyes naturally look at, or are ‘drawn’ to.

The oculometer (5) detects the viewing direction (14) and passes thisinformation to the processor (9) which determines points of‘eye-fixation’ by measuring the time-duration in which the user's eyesare looking at a part (13) of the first image (11). Algorithms areapplied to determine whether that time-duration is sufficient for thatpart (13) of the image (11) to be considered an ‘area-of-interest’.

When an ‘area-of-interest’ is detected, the processor (9) passes signalsto the second display screen (10) to increase the brightness andcontrast of the image part (13) to thus increase clarity.

The size of the part (13) of the first image (11) identified as an‘area-of-interest’ is dependant on the output signals from theoculometer (5) i.e. the extent of saccadic eye-movement of the user (6)defines the size of the image part (13) considered an‘area-of-interest’.

Thus, by simply looking at the image (11), the user (6) can clarify theparts (13) of the image (11) as it interests them. With the controlbuttons (not shown) the user (6) can move through the pre-selected setof images or control a video feed.

Simultaneous with this identification of ‘areas-of-interest’ is thecopying and display of the part (13) of the image (11) as an image part(15) of second image (12) on the first display screen (2). The firstdisplay screen (2) is remote to the second display screen (10) and isnot visible to the user (6). The first display screen (2) is howevervisible to an audience or another participant (not shown). The remotedisplay screen (10) provides a ‘default’ background image or darkeneddisplay onto which the ‘areas-of-interest’ (13, 15) are displayed.

The parameters, e.g. size, intensity, contrast, clarity, coherence andcolor of the part (13), of the first image (11) copied and displayed asimage parts (15) are dependant on the output signals from thebio-feedback sensors (4) (shown in FIG. 1 only).

Thus, the intensity and color of image part (15) is dependant on a levelof ‘excitation’ of the user (6) as determined by the processor (9) fromthe signals from the bio-feedback sensors (4) detecting the galvanicskin response or heart-rate of the user (6).

The parts (15) of the remotely displayed image (12) are typically fadedtowards the edge of the part (15) to provide an enhanced visual effect.

The size of the image part (15) displayed corresponds partially to thesize of the ‘area-of-interest’ or part (13) (i.e. proportional to theextent of saccadic eye-movement of the user (6)) and due to the level of‘excitation’ of the user (6) as determined by the processor (9) from thesignals from the bio-feedback sensors (4).

The only image seen on the remote first display screen (2) is a secondimage (12) constructed by the copying of ‘areas-of-interest’ (parts(13)) selected by the user (6). The areas of the image (11) that are notidentified as ‘areas-of-interest’ are not displayed on the remote screen(2). The parts (15) of the remote image (12) are displayed for a settime-period proportional to the level of relaxation or excitation of theuser (6) as detected by the bio-feedback sensors (4) when looking at thecorresponding first image part (13). The areas-of-interest (15) may alsobe retained as a visual record of visual and physiological activity onthe part of the user.

The image parts (15) appear and fade-away as the user (6) changes thepoint of eye-fixation on the first image (11) thus creating a‘real-time’ record on the audience display screen (2) of where the user(6) is looking at, and the user's visual ‘areas-of-interest on the firstimage (11).

When a user (6) looks at a part (13) of an image repeatedly, thecorresponding part (15) of image (12) will be reinforced.

The image (12) on the audience display screen (2) is thus a compositeimage formed by parts (13) of image (11) copied onto the display (2) insaccadic movements corresponding to the movements of the user's eyes.Merged or overlapped copied parts (15) which have the potential to reacha point of visual ‘saturation’ are controlled by averaging orinterpolating pixel values so as not to exceed pre-establishedthresholds and render the image (12) indiscernible.

FIG. 3 shows a process diagram of a method of operating the imagegeneration system (1) as shown in FIG. 1 or FIG. 2. These processes arealso shown in FIG. 1.

After calibration (100), a first image (11) is presented (101) to a useron a second display screen (10). The user (6) decides (102) whether thefirst image (11) is to remain or be changed (103) by choosing anotherimage by cycling through pre-selected images either randomly or in apredetermined sequence.

The processor (9) determines (104) an orientation of the user's eyesfrom the signals from the eye-tracking system (3) and therefore also apoint of eye-fixation (14) on the first image (11).

The processor (9) determines the size, (process 105) of the part (13) ofthe image (11) to be copied and the parameters, (process 106) of thecopied image part (15) to be displayed on the display screen (2)indicative of the input (108) received from the bio-feedback sensors(4).

The processor (9) then sends (107) signals to the display screen (2) todisplay the copied image part (15).

The steps 101-107 are repeated iteratively. The image (12) displayed(107) will also change as the user (6) looks at different portions ofthe first image (11) and/or changes (103) the first image (11).

System diagrams of two preferred embodiments of the image generationsystem (1) are shown in FIGS. 3 and 4. FIG. 3 shows a system diagram ofthe image generation system (1) of FIGS. 1-3 while FIG. 4 shows analternative embodiment (1 a) where the first (11) and second (12) imagesare displayed on the same display screen (10). The system (1 a) shown inFIG. 5 is also used in applications where the second image (12) isformed by modifying the first image (11) in response to the bio-feedback(108) and/or eye-tracking (104) inputs.

It will be readily apparent to one skilled in the art that the imagegeneration system (1) has particular application in computer gaming,e.g. the combination of bio-feedback (108) and eye tracking (104)measurements can be used as controls for manipulating the visual gameenvironment.

Reference herein to “computer games” should be understood to refer toall interactive computerized systems utilizing a display screen and byway of example may include: computer or gaming console software,gambling or ‘slot’ machines, vending machines or the like.

Computer games often present game participants with scenarios andcharacters or ‘avatars’ within an interactive virtual environment. Thesevirtual environments present a visual space within which the user (6)interacts with other users and/or computer generated elements.

The image generation system (1) can use the eye-tracking system (3) andbio-feedback sensors (4) systems to determine and monitor where the user(6) is looking within the virtual environment and measure thebio-feedback (108) to attribute a measurement of a physiologicalfunction to the parts of the virtual environment that are looked at.Thus, the user (6) can interact with the environment and/or theenvironment can be programmed to change in response to the bio-feedback(108) and eye-tracking (104) response.

The applications for the image generation system (1) in computer gamesare many and varied. For example, the image generation system (1) may beused in any application from use as a relatively simple design orcontrol tool for visual changes, to a more complex interactive controlwhere the generation and navigation of a virtual environment iscontrolled by the learned coordination of simultaneous eye movement andbody state.

A theoretical example of using the image generation system (1) as such acomplex control may be in an interactive “slip stream” event such as inthe television series “Andromeda” where the user (6) controls theenvironment by coordination of eye-movement and body-state.

Another exemplary computer game using such a control may involve travelof a user's (6) ‘avatar’ through a virtual ‘tunnel’. The user/avatar isguided by image modulations that visually reflect the bio-feedbackmeasurements (108). The user (6) maintains a directed movement “through”the visual space by consciously controlling or responding to images (12)generated representing the bio-feedback (108) while simultaneouslydirecting the user's (6) eye-fixation (14) toward a visual objectivethat provides a reward or achieves a goal, e.g. continuing the movementor selecting an exit. The computer game may be configured such that ifthere is an involuntary or unconscious change, e.g. where the user (6)enters a changed body state or the user's eyes are distracted, the user(6) may find their avatar deposited in a different and possiblyunexpected environment with a new ratio of positive and negative sets ofattributes for engagement.

An example of a computer game using the image generation system (1) isshown in FIG. 6 and is generally indicated by arrow 200. The computergame (200) is configured to control the processor (9) to present visualand/or auditory information to the user (not shown in FIG. 6) and theuser responds using inputs (201) including direct input (202), changesin eye-orientation (203) and/or bio-feedback (204). The processor (9)receives the inputs (201) and accordingly modifies the informationpresented to the user (6) according to one or more of the gamealgorithms. While not shown in FIG. 6, the information is presented viaa display screen (10) and speakers (17) as shown in FIG. 1.

The computer game (200) can thus be controlled by the user to change anavatar's attributes (e.g. emotions, skills etc.) to correspond with apossible selection of avatar entities (205), including “Lover”,“Magician”, “Warrior” and “King”. The user can only transition to afinal entity (206) (“Fool”) after reaching one or more of the earlierentitles (205) or by having such control over their bio-feedback (204)and/or eye-orientation (203) that they can transition directly from thestart. Once in the “fool” entity (206) the user can transition toanother level of engagement or finish and complete the game. Thecomputer game (200) is also configured to introduce or posit visualelements or objects, (e.g. weapons, magic items, characters etc.) intothe environment in response to preset algorithms or to changes ineye-orientation (203) or bio-feedback (204).

In another computer game the image generation system (1) may be used asa virtual movement or manipulation tool where the user (6) moves througha virtual environment looking for “enemies”, e.g. as in a combatsimulation. The computer game is configured such that when the user (6)“looks” around and moves through the environment they produce differentunconscious responses (bio-feedback (108)) to elements in theenvironment. As they have these experiences, the response (108) triggersvisual changes within the environment or may open a ‘portal’ or ‘window’to a parallel environment with a different set of decision trees thatare discontinuous with the “logic” of the previous or currentenvironment.

An example of the generation of a portal or window of such a computergame is shown in FIG. 7. The portal or window is generated by using theimage generation system (1) to create a transitional phase where partsof a second image (12) are superimposed on the image presented to theuser, e.g. the first image (11), depending on the area of the firstimage (11) corresponding to the location of the user's eye-fixation(14). The second image (12) represents another virtual environment,entity or element.

The second image (12) will grow in size, clarity or brightness if theeye-fixation is maintained and, depending upon whether the user (6)reaches a particular parameter set of combined eye-tracking (104) andbio-feedback (108) responses, the visual transition will either fadeback into the first image (11) or continue to grow to displace the firstimage (11). Thus, the second image (12) may provide a visual “portal”effect, e.g. as shown in FIG. 7, by generating such visual “swatches”(18) of the second environment that appear in the form of a “rain” or“sizzle” of swatches (18) which increase in number and/or size toresolve into the second computer generated environment.

It should also be appreciated that the bio-feedback (108) and/oreye-tracking (104) responses may be used to alter the visualrepresentation of the avatar to other players of the game, e.g. otherplayers may see a different visual representation of the avatar than theuser controlling the avatar. The physiological state of each player maythus be represented as a visual alteration of the players' avatarsthereby providing another dimension to interactive computer gaming. Inone embodiment the visual alteration may be a change in the facialexpression of the avatar.

The visual representation of an avatar can thus provide a source ofvisual information within a display, e.g. acting as a display within adisplay. Such an embodiment can be useful in alternate reality worldsfor representing the emotions of users which, in the real world would beprovided as facial expressions. The representation of emotional statesmay also be useful in digital communications (e.g. email) by attachingan indicator representing the emotional state of the sender, thusproviding a greater level of information to the receiver.

The following examples are various other applications for which thepresent invention has particular use. However, it will be appreciated byone skilled in that art that the principles of the present invention hasapplication in many technological fields and the examples herein shouldnot be seen to be limiting.

The present invention is useful in the assessment of the efficacy ofinformation presentation in, for example, advertising, education, andpromotions. This assessment is achieved by using the image generationsystem (1) to assess correlations between the combined response from theeye-tracking (3) and bio-feedback (16) systems with the effectiveness ofinformation delivery in terms of information retention, recall, andcomprehension. In such an application the image generation system (1)will be generally the same to that shown in FIGS. 1 and 2. The processor(9) however may be programmed to process the bio-feedback (108) andeye-tracking (104) responses to provide data and/or visual informationto the display screen (2) rather than generate a composite image (12).The data and/or information provides an educator with measures of theuser's (6) response to visual and/or and auditory information presentedto the user (6).

Education utilizing computers and associated software can be engaging toa degree, though the uniform delivery of the materials lacks theemotional interactions that a teacher can use to keep a studentmotivated and on-task. Repeated prompting and encouragement generated bythe computer software may leave the student disinterested and theprompts may prove ‘empty’ and ‘programmed’.

Thus, Instructional designers, especially those developing distanceeducation interactions, can utilize the image generation system (1) tomonitor the effectiveness of educational design materials, i.e. bymonitoring the eye-tracking (104) and bio-feedback (108) responses toeducational stimuli. A generalized measure of the educational experiencecan be derived for use in instructional re-designs by correlating theresponses of multiple users to the same tasks and materials. The imagegeneration system (1) can also show such responses to the user in ‘realtime’, thereby providing a dynamic investigation tool. The bio-feedback(108) and eye-tracking (104) responses may be used, for example, toindicate areas needing further attention.

Another application for the image generation system (1) is in providingvisual indications of the effectiveness of visual designs, e.g.advertising and promotional images or video and an example is shown inFIG. 8. The image generation system (1) includes the first (2) andsecond (10) display screens, as also shown in FIG. 1. The second displayscreen (10) presents an image (11) to the user (6). The processor (9)measures the signals from the bio-feedback (16) and eye-tracking (3)systems to determine the ‘areas-of-interest’ (13) and the associatedphysiological response. The parts (13) of the image (11) identified as‘areas-of-interest’ are superimposed on the first display screen (2) asparts (15) and indicate those portions that attract the user's (6)attention and those that evoke a physiological response. The parts (13)that evoke a physiological response are indicated by brighter and/orless diffuse parts (15) of the image (12). The image generation system(1) can thus be used to establish correlations between the visual andaffective associations of visual design with identification recognitionand targeted product associations. The image generation system (1) canalso be used in studying the visual effectiveness of any kind of visualdisplay to deliver information that is retained by the user, whether theinformation is educational or promotional.

The image generation system (1) may also find use in psychologicalprojection testing, i.e. tests like the Thematic Apperception Test (TAT)that are administered, scored, and interpreted by a therapist or othertester who requests a subject talk about what is shown in a set ofimages.

In a TAT the responses to the testing are recorded (written, audio orvideo) and assessed based upon the experiences and training of thetester and supporting instructional documentation for the test.

The image generation system (1) shown in FIGS. 1 and 2 can be used tosupport such testing by monitoring the eye-fixation (14) of the user (6)to identify the ‘areas-of-interest’ in the image (11) presented. Thus,by correlating the areas-of-interest with the corresponding “emotional”(physiological) responses measured by the bio-feedback sensors (7), thetester can determine the emotional response to particular images orparts thereof and therefore create a possibly more ‘complete’ account ofthe user's (6) responses. The image generation system (1) can therebypotentially provide a more objective recording of the testing, ratherthan relying on the user's (6) verbal responses to the images (11) or toa particular tester's interpretation of such responses.

Such an image generation system (1) can also be used to train military,law enforcement, emergency service personnel or persons who are exposedto graphically violent or disturbing environments. The image generationsystem (1) can be used to measure a user's responses to images presentedand thereby facilitate training of the user to control theirphysiological response when presented with adverse environments.

Yet another application for the image generation system (1) is inuser-interaction analysis and usability testing. The informationgarnered from usability and interaction analysis using the imagegeneration system (1) is potentially more useful for many types ofanalysis as the bio-feedback (108) and eye-tracking (104) responses arecoupled together to generate a visual image (12) of the user's (6)response and interaction with the testing environment. The combinationof eye-tracking (104) and bio-feedback (108) to generate correspondingimages (12) can be correlated with assessments of the effectiveness ofvisual communications in delivering information, or, as in the case of acomputer interface or web-page, the usability and navigability.

FIG. 9 shows yet another embodiment of a computer ‘game’ using thepresent invention. This game (300) is a blackjack game run from a serveron a computer network such as the Internet. It will be appreciated thatthe general aspects of this embodiment may be used with any computergame on which an avatar is used, for example, the principles of thisembodiment may be used to communicate the physiological and emotionalstate of users in virtual ‘worlds’ or MMORPGs.

It will be appreciated that the computer program and game parametersused in computer blackjack, gambling and other games are well known inthe art and will not be described herein.

FIG. 9 shows a game GUI (301) that provides an interface with thecomputer game Application Program Interface API. This GUI (301) isdisplayed on a display screen (10) and has images (303 a-303 c) ofavatars representing other players (6 a-6 c) on the network. In turn theuser (6) has an avatar (not shown) that is similarly displayed to theother players.

The GUI (301) also includes images of game items provided in the form ofplaying cards (305 a-305 d) held by each player. Each player has animage generation system (1) connected to the server running the program(300) via the user's computer system. The image generation system (1)provides bio-feedback (108) and eye-tracking (104) data to the serverand this data (104, 108) is processed using eye-tracking, bio-feedbackand image generation algorithms to generate an avatar image (otherusers' avatars are shown as 303 a, 303 b, 303 c) representing thephysiological state of the particular user as facial expressions,coloring or other indicators on the avatar image. The eye-tracking datais used to generate arrows (304 b, 304 c) indicating the viewingdirection of the users (6 a-6 d), though in FIG. 9 arrows are only shownfor users (6 b) and (6 c).

Online blackjack and other competitive games provide no way in whichusers can view the facial expressions, emotions and other physiologicalstates of their competitors. In contrast, real games provide facialexpressions and other body language as cues on the emotive state o acompetitor. The embodiment shown in FIG. 9 similarly provides a visualrepresentation of the physiological state of the user as well asindicating what the user is looking at.

Aspects of the present invention have been described by way of exampleonly and it should be appreciated that modifications and additions maybe made thereto without departing from the scope of the appended claims.

1. An image generation system for generating an image on a displayscreen, the image generation system including: an eye-tracking systemcapable of determining a user's eye orientation and outputting a signalindicative of same; a bio-feedback sensor capable of detecting activityof one or more physiological functions of the user and outputting asignal indicative of the level of activity; and a processor adapted toreceive and process the output signals from the eye-tracking system andbio-feedback sensor; wherein the processor determines an image to begenerated on the display screen indicative of the signals from theeye-tracking system and bio-feedback sensor.
 2. An image generationsystem as claimed in claim 1, wherein the eye-tracking system is capableof detecting eye-movement of a user.
 3. An image generation system asclaimed in claim 1, wherein the processor processes the signal from theeye-tracking system to determine eye-fixation, eye-fixation being thetime-duration in which a user's eyes are directed in one or more viewingdirections from the detected eye-orientation.
 4. An image generationsystem as claimed in claim 1, wherein the processor determines alocation to display a part of the generated image from the output signalfrom the eye-tracking system.
 5. An image generation system as claimedin claim 1, wherein the processor varies at least one parameter of apart of the image to be displayed on the display screen from the outputsignal from the bio-feedback sensor.
 6. An image generation system asclaimed in claim 1, wherein the processor determines a sound to begenerated from a speaker indicative of the signals from the eye-trackingsystem or the bio-feedback sensor.
 7. An image generation system asclaimed in claim 1, wherein the eye-tracking system and/or bio-feedbacksensor are adapted to be substantially portable and capable of remotelytransmitting a signal to the processor.
 8. An image generation system asclaimed in claim 1, wherein the eye-tracking system includes anoculometer capable of detecting the orientation of an eye.
 9. An imagegeneration system as claimed in claim 8, wherein the oculometer ismovable on a mounting such that it may be aligned with a user's eyes.10. An image generation system as claimed in claim 8, wherein theoculometer is automatically movable to maintain alignment with a user'seyes.
 11. An image generation system as claimed in claim 8, wherein theoculometer is coupled to a user-location system, the user-locationsystem capable of identifying the location or orientation of the userand determining the position of the user's eyes by extrapolating fromthe user location.
 12. An image generation system as claimed in claim 1,wherein the bio-feedback sensor includes one or more sensors capable ofmeasuring at least one of a user's galvanic skin response, heart-rateblood pressure, cerebral electric potential, skin temperature, muscletension, heart-rate variability, blood-pressure variability, skinmoisture.
 13. An image generation system as claimed in claim 1,including a processor for each of the eye-tracking system and thebio-feedback sensor, said processors being programmed to process theoutput signals from the eye-tracking system and bio-feedback sensor fortransfer to a digital computer, the digital computer including aprocessor capable of determining an image to be generated on the displayscreen indicative of the signals from the eye-tracking system andbio-feedback sensor.
 14. An image generation system as claimed in claim1, including: a first display including the display screen fordisplaying the generated image; a second display for presenting a firstimage to the user; wherein the eye-tracking system is capable ofdetermining one or more points of eye-fixation of a user on the firstimage, the processor determining an image to display on the firstdisplay indicative of the signals from the eye-tracking system andbio-feedback sensor.
 15. An image generation system as claimed in claim14, wherein the processor determines a position of a part of the imageto be displayed on the first display from the output signal from theeye-tracking system.
 16. An image generation system as claimed in claim14, wherein the processor determines a parameter of a part of the imageto be displayed on the first display from the output signal from theeye-tracking system.
 17. An image generation system as claimed in claim14, wherein the processor determines a parameter of a part of the imageto be displayed on the first display from the output signals from theeye-tracking system indicative of a level of saccade.
 18. An imagegeneration system as claimed in claim 14, wherein the image determinedto be displayed on the first display is an at least partial copy orrepresentation of the first image.
 19. An image generation system asclaimed in claim 18, wherein the image determined to be displayed on thefirst display is an at least partial copy or representation of a part ofthe first image corresponding to the user's eye-fixation on the firstimage.
 20. An image generation system as claimed in claim 19, whereinthe size of the part of the first image copied and displayed on thefirst display is dependant on the output signal from the eye-trackingsystem.
 21. An image generation system as claimed in claim 14, whereinthe second display includes a second display screen, the first imagebeing displayed thereon.
 22. An image generation system as claimed inclaim 14, wherein the first and second displays are part of a singledisplay screen.
 23. An image generation system as claimed in claim 14,wherein the first image is a three-dimensional image of avirtual-reality environment and a control system is provided to navigatethrough the virtual-reality environment and change the first image. 24.An image generation system as claimed in claim 23, wherein the controlsystem may include the eye-tracking system and bio-feedback sensor. 25.An image generation system as claimed in claim 14, wherein the processoris connected to the second display and programmed to present the firstimage as filtered, processed, distorted and/or rendered at leastpartially unclear.
 26. An image generation system as claimed in claim25, wherein the processor is programmed to vary at least one imageparameter of a part of the first image to improve image clarity when theeye-tracking system detects the user looking at said part.
 27. An imagegeneration system as claimed in claim 14, wherein the processordetermines a parameter of a part of the image to be displayed on thefirst display from the output signal from the bio-feedback sensors. 28.An image generation system as claimed in claim 27, wherein the size ofthe part of the image displayed on the first display is dependant on theoutput signal from the bio-feedback sensor.
 29. An image generationsystem as claimed in claim 14, wherein the second display includesphysical objects or images, the first image being formed therefrom. 30.An image generation system as claimed in claim 29, including an imagecapture device for capturing the first image as a digital image.
 31. Acommunication system for allowing communication between two users at twoseparate user locations, the communication system including two imagegeneration systems as claimed in claim 30, wherein each said imagegeneration system is operable to generate an image on a viewer displayscreen and each image generation system includes: a said eye-trackingsystem; a said bio-feedback sensor; a said processor adapted to receiveand process the output signals from the eye-tracking system andbio-feedback sensor, and a said second display for presenting a firstimage to the user, the second display connected to the image capturedevice, wherein the eye-tracking system is capable of determining one ormore points of eye-fixation of a user on the first image, the processordetermining an image to display on the display screen indicative of thesignals from the eye-tracking system and bio-feedback sensor, and ateach of said user locations, the bio-feedback sensor, eye-trackingsystem, processor and second display screen of one image generationsystem, and the viewer display screen and image capture device of theother said image generation system are located, and each image capturedevice is configured to capture an image and display said captured imageon the second display at the other location, each processor isconfigured to determine an image to display on the viewer display screenat the other user location indicative of the signals from theeye-tracking system and bio-feedback sensor.
 32. The communicationsystem of claim 32, wherein the image capture device is a video cameraadapted to capture an image of the user at the respective location.